Text mining and natural language processing for the early stages of space mission design

Final thesis submitted December 2021 - degree awarded in 2022A considerable amount of data related to space mission design has been accumulated since artificial satellites started to venture into space in the 1950s. This data has today become an overwhelming volume of information, triggering a significant knowledge reuse bottleneck at the early stages of space mission design. Meanwhile, virtual assistants, text mining and Natural Language Processing techniques have become pervasive to our daily life. The work presented in this thesis is one of the first attempts to bridge the gap between the worlds of space systems engineering and text mining. Several novel models are thus developed and implemented here, targeting the structuring of accumulated data through an ontology, but also tasks commonly performed by systems engineers such as requirement management and heritage analysis. A first collection of documents related to space systems is gathered for the training of these methods. Eventually, this work aims to pave the way towards the development of a Design Engineering Assistant (DEA) for the early stages of space mission design. It is also hoped that this work will actively contribute to the integration of text mining and Natural Language Processing methods in the field of space mission design, enhancing current design processes.A considerable amount of data related to space mission design has been accumulated since artificial satellites started to venture into space in the 1950s. This data has today become an overwhelming volume of information, triggering a significant knowledge reuse bottleneck at the early stages of space mission design. Meanwhile, virtual assistants, text mining and Natural Language Processing techniques have become pervasive to our daily life. The work presented in this thesis is one of the first attempts to bridge the gap between the worlds of space systems engineering and text mining. Several novel models are thus developed and implemented here, targeting the structuring of accumulated data through an ontology, but also tasks commonly performed by systems engineers such as requirement management and heritage analysis. A first collection of documents related to space systems is gathered for the training of these methods. Eventually, this work aims to pave the way towards the development of a Design Engineering Assistant (DEA) for the early stages of space mission design. It is also hoped that this work will actively contribute to the integration of text mining and Natural Language Processing methods in the field of space mission design, enhancing current design processes

From engineering models to knowledge graph : delivering new insights into models

Essential information on the early stages of a mission design is contained in Engineering Models. Yet, these models are often uneasy to visualise, query, let alone compare. This study demonstrates how Knowledge Graphs can overcome these data silos, interconnect information, provide a big-picture perspective, and infer new knowledge that would have remained hidden otherwise. Following the migration of CubeSats Engineering Models to a Knowledge Graph, two case studies are explored. The first case study illustrates how graph inference can derive implicit knowledge from existing explicit concepts. In the second case study, a Natural Language Processing layer is adjoined to the Knowledge Graph to enhances the analysis of textual content. The Natural Language Processing layer relies on the document embedding method doc2v

Concurrent engineering and social distancing 101 : lessons learned during a global pandemic

Social distancing measures introduced in the wake of COVID-19 greatly impacted the concurrent engineering process. This paper addresses methodological adaptation measures which are required to ensure the continuity of this activity. Two CubeSat feasibility studies run at the University of Strathclyde, one physical and one virtual, are compared to quantify the impact of the adaptation. Three evaluation criteria are used: the fulfilment of requirements & customer satisfaction, server data flow rate and participant perceptions. The results indicate that although adaptation was successful, it failed to lift all communication barriers introduced by virtual exchanges

SEA from Space - Coastal management via satellites : strategic planning in response to global sea level rises

Rising global sea levels are already having an extremely damaging effect on local communities, human infrastructure, and the economy. The negative social and economic repercussions of this is only expected to worsen, meaning that effective coastal management strategies are critical. This project proposes the development of a new tool which acts as a strategic decision-support system for coastal management. The tool will make use of satellite data to characterise shoreline pressures and impacts in terms of future flood potential and coastal erosion, with a view of generating a solution for the most applicable and cost-effective coastal management strategy for different shorelines. These results can then be used to inform Shoreline Management Plans

STRATHcube : The design of a student CubeSat using concurrent engineering methods

With the role of concurrent engineering (CE) becoming more important to the success of companies, it is vital that engineering students are able to understand and apply this concept. In this regard, the University of Strathclyde regularly offers its students opportunities to learn about this process through practical-based CE workshops. The results from a student-based CE study of a CubeSat are therefore outlined, including the effectiveness of the session as a learning experience for students. Through collaboration and teamwork, the student team produced a feasible design concept which achieved most of the prespecified objectives. Additionally, it was determined that the learning outcomes of the study were widely met, despite it taking place virtually due to COVID-19

Natural language processing for explainable satellite scheduling

Facilitating the interactions between humans and Artificial Intelligence (AI) in automated systems is becoming central with the advancements in technology and their more widespread adoption in practical applications. Mathematical programming scheduling techniques are a driving factor to assist ground station operators both on board the satellite, for autonomous decision making, and on ground, for supporting mid-term operations scheduling. When communication to ground is limited, scheduling algorithms require a level of autonomy and robustness able to respond to issues arising on board the satellite in the absence of communication with a ground operator. Moreover, explanations must be generated, along side schedules, for the operator to build and gain trust in the autonomous system. Explainable Artificial Intelligence (XAI) is an emerging topic in AI. Explanations are a necessary layer to effectively deploy autonomous trustworthy systems in practical applications. Queries may arise from operators such as why, what, when and how the scheduled actions were selected autonomously on board for a specific time. Explanations are provided based on the definition of the problem with its respective constraints. Autonomous decision making algorithms can be explained in several ways. Computational Argumentation (CA) and Natural Language Processing (NLP)) are some techniques, belonging to the domains of formal logic and machine learning, that can be used to generate explanations and communicate them back to the user in the form of textual output. An Argumentation Framework (AF) was created to assist in answering questions raised by the end user. The AF encodes, in its lower level, all the necessary information on when conflicts may occur between actions, as well as, environmental conditions inhibiting the occurrence of the actions within a schedule. This database of information is used to construct arguments in support or negation of user submitted queries or to provide an explanation of the complete derived schedule. NLP is then used as a bridge to communicate the relevant arguments to the user. The queries received revolved around three main areas: the subject, the time of interest and the intent. Following the interpretation, the queries were mapped to the AF database, returning a list of conflicts, agreements and neutral outcomes. The chosen NLP method for this architecture, GPT-3 was used to then deduce the answer to the query and justify it with a textual explanation

Space transformers : language modeling for space systems

The transformers architecture and transfer learning have radically modified the Natural Language Processing (NLP) landscape, enabling new applications in fields where open source labelled datasets are scarce. Space systems engineering is a field with limited access to large labelled corpora and a need for enhanced knowledge reuse of accumulated design data. Transformers models such as the Bidirectional Encoder Representations from Transformers (BERT) and the Robustly Optimised BERT Pretraining Approach (RoBERTa) are however trained on general corpora. To answer the need for domain specific contextualised word embedding in the space field, we propose Space Transformers, a novel family of three models, SpaceBERT, SpaceRoBERTa and SpaceSciBERT, respectively further pre-trained from BERT, RoBERTa and SciBERT on our domain-specific corpus. We collect and label a new dataset of space systems concepts based on space standards. We fine-tune and compare our domain-specific models to their general counterparts on a domain-specific Concept Recognition (CR) task. Our study rightly demonstrates that the models further pre-trained on a space corpus outperform their respective baseline models in the Concept Recognition task, with SpaceRoBERTa achieving significant higher ranking overall

A system engineering recommendation system based on language similarity analysis : an application to space systems conceptual design

In Model-Based System Engineering (MBSE), the creation of complex engineering systems is facilitated by a standard- ised engineering data model and model version control, both of which generate valuable data after each conducted study. However, there are currently few to no approaches, reusing the information and knowledge from previous engineering studies. In this work, we present a new recommendation system, based on a widely adopted engineering data model, defined in the ECSS-E-TM-10-25A technical memorandum. This engineering data model is used by the European Space Agency (ESA), associated partners, as well as in other engineering domains. An engineering model (EM) is a hierar- chical decomposition of an engineering system, providing information about the overall system, design options but also about low-level components. The novel recommendation system leverages a Knowledge Graph (KG) as a unified frame- work for storing multiple EMs. State-of-the-art semantic similarity Natural Language Processing (NLP) techniques are then used to define similarity between higher level information, so called metadata, associated with each EM. Textual information, such as the ”Mission Objectives” of each study, are encoded with a neural language model into a vector representation, which allows to calculate a similarity metric between them, and then compare past-mission metadata with proposed metadata of a new study. In addition, a similarity between lower-level engineering components in the KG is described through the Jaccard metric, which compares components based by the set of parameters that each of them are associated with. By firstly clustering similar engineering designs through their associated metadata and then identifying analogous components in each cluster, the algorithm is able to recommend engineering components for new studies. In the results, the functionality of the approach is demonstrated as a pilot study for spacecraft conceptual design

The automatic categorisation of space mission requirements for the Design Engineering Assistant

To enhance Knowledge Reuse in the field of space mission design, the implementation of Information Retrieval (IR) is key. Topic Modeling (TM) is used to identify, learn and extract topics from a corpus of documents, and can therefore support several IR tasks such as categorisation. This study relies on a common TM method, Latent Dirichlet Allocation (LDA), a probability-based approach. An extensive Wikipedia-based corpus focused on space mission design is collected, parsed, preprocessed, and used to train a general ’Space Mission Design’ LDA model. The LDA model is optimised based on the perplexity measure for a range of topics numbers. The topics dictionaries of the retained model are labelled by human annotators, with labels corresponding to spacecraft subsystems. The performances of the general model are evaluated against a set of space mission requirements with a categorisation task. The general model is then used as a base to generate specific LDA models focused on one topic, or spacecraft subsystem. The general LDA model developed in this study proves to be a solid base for the generation of focused LDA models, yielding very high accuracy scores and Mean Reciprocal Ranking.Finally, a semi-supervised LDA model, fed with lexical priors is trained, leading to improved performances of a general mode

Artificial intelligence for the early design phases of space missions

Recent introduction of data mining methods has led to a paradigm shift in the way we can analyze space data. This paper demonstrates that Artificial Intelligence (AI), and especially the field of Knowledge Representation and Reasoning (KRR), could also be successfully employed at the start of the space mission life cycle via an Expert System (ES) used as a Design Engineering Assistant (DEA). An ES is an AI-based agent used to solve complex problems in particular fields. There are many examples of ES being successfully implemented in the aeronautical, agricultural, legal or medical fields. Applied to space mission design, and in particular, in the context of concurrent engineering sessions, an ES could serve as a knowledge engine and support the generation of the initial design inputs, provide easy and quick access to previous design decisions or push to explore new design options. Integrated to the User design environment, the DEA could become an active assistant following the design iterations and flagging model inconsistencies. Today, for space missions design, experts apply methods of concurrent engineering and Model-Based System Engineering, relying both on their implicit knowledge (i.e., past experiences, network) and on available explicit knowledge (i.e., past reports, publications, data sheets). The former knowledge type represents still the most significant amount of data, mostly unstructured, non-digital or digital data of various legacy formats. Searching for information through this data is highly time-consuming. A solution is to convert this data into structured data to be stored into a Knowledge Graph (KG) that can be traversed by an inference engine to provide reasoning and deductions on its nodes. Knowledge is extracted from the KG via a User Interface (UI) and a query engine providing reliable and relevant knowledge summaries to the Human experts. The DEA project aims to enhance the productivity of experts by providing them with new insights into a large amount of data accumulated in the field of space mission design. Natural Language Processing (NLP), Machine Learning (ML), Knowledge Management (KM) and Human-Machine Interaction (HMI) methods are leveraged to develop the DEA. Building the knowledge base manually is subjective, timeconsuming, laborious and error bound. This is why the knowledge base generation and population rely on Ontology Learning (OL) methods. This OL approach follows a modified model of the Ontology Layer Cake. This paper describes the approach and the parameters used for the qualitative trade-off for the selection of the software to be adopted in the architecture of the ES. The study also displays the first results of the multiword extraction and highlights the importance of Word Sense Disambiguation for the identification of synonyms in the context. This paper includes the detailed software architecture of both front and back-ends, as well as the tool requirements. Both architectures and requirements were refined after a set of interviews with experts from the European Space Agency. The paper finally presents the preliminary strategy to quantify and mitigate uncertainties within the ES